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Agenda 


□  Situation  Awareness  versus  Situated  Cognition 

□  A  Process  Model  of  Situated  Cognition 

□  The  USS  Stark:  A  Case  Study  (Individual) 

□  Team  versus  Shared  Situation  Awareness 

□  The  USS  Stark:  A  Case  Study  (Distributed) 

□  Measurement  Methods  and  Metrics 


X - ^ - N 

^  “The  perception  of  elements  in  the  ^ 

environment  within  a  volume  of  time 
and  space,  the  comprehension  of 
their  meaning,  and  their  status  in 
the  near  future.”  (Endsley,  1988) 


“A  common,  relevant  picture  of  the 
battlefield  scaled  to  specific  levels  of 
interests  and  special  needs." 

(TRADOC  Pamphlet  525-5) 


“The  product  of  applying  analysis  and 
judgment  to  the  common  operational 
picture...”  (FM  3-0  (Operations)) 

Ideal  SA;  Achievable  SA;  Actual  SA 

(Pew,  2000) 

“Where  am  I?  Where’s  my  buddy? 
Where’s  the  enemy?”  (An  Army  Officer) 


“That’s  my  SA  (pointing  to  his  FBCB2 
^screen).” _  (An  Enlisted  Soldier) 


Methods  for  Measuring  SA 


Subjective 

Prospective 

Direct 

Obtrusive 


◄ - ► 

◄ - ► 

◄ - ► 

◄ - ► 


Objective 

Retrospective 

Indirect 

Unobtrusive 


□  SART:  Situational  Awareness  Rating  Technique 

□  SA-SWORD:  Situation  Awareness-Subjective  Workload  Dominance 

□  SARS:  Situation  Awareness  Rating  Scale 

□  MARS:  Mission  Awareness  Rating 

□  SAGAT:  Situational  Awareness  Global  Assessment  Technique 

□  SALIENT:  SA  Linked  Instances  Adapted  to  Novel  Tasks 

□  SABARS:  Situation  Awareness  Behaviorally  Anchored  Rating  Scale 


These  methods  tend  to  measure: 

□  states,  not  processes 

□  humans,  not  systems 


An  Alternative  to 
Situation  Awareness 

What  is  needed  is  a  model  and  a  methodology  that: 

□  focuses  on  processes  rather  than  states 

□  includes  both  human  and  machine  ‘components’  of  a 
system 

□  is  oriented  on  assessing  human-system  performance 

□  tracks  the  evolution  of  activities  and  cognition 


Situated  Cognition 

□  Borrowed  from  the  learning  and  linguistics  literature 

□  Includes  mental  activities  embedded  in  an  evolving 
context 

□  Includes  human  and  machine  agents 

□  Involves  collaborative  activities 


□  Goal-directed 


A  Process  Model  of  Situated  Cognition 


Technological 

Systems 


Perceptual  and 
Cognitive  Systems 


Projection  of 
decision  maker 


©  Miller  and  Shattuck,  2003 


Misshaped  lenses  will  skew  a  decision  maker’s 
perceptions,  comprehensions,  and  projections 


©  Miller  and  Shattuck,  2003 


A  Process  Model  of  Situated  Cognition 
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©  Miller  and  Shattuck,  2003 


Feedback  Loops  in  the  Model 


Data 
Detected 
By 
Sensors 


©  Miller  and  Shattuck,  2003 


On  the  evening  of  May  17,  1987,  the  USS  Stark  was  patrolling  international 
waters  in  the  Persian  Gulf  off  the  coast  of  Bahrain  and  Saudi  Arabia.  At 
2109  that  evening,  the  USS  Stark  was  struck  by  the  first  of  two  Exocet  AM- 
39  anti-ship  cruise  missiles,  fired  from  an  Iraqi  F-1  Mirage  fighter. 


Oval  1:  Ground  Truth 


Oval  1:  Ground  Truth 

Stark  is  12nm  west  of  Iraqi 


exclusion  zone. 


Oval  1:  Ground  Truth 


AWACS  is  aloft. 


Oval  1:  Ground  Truth 

Other  USN  vessels  are  in  the  area. 


Oval  1:  Ground  Truth 

Iraqi  F-1  Mirage  flying  toward  the  USS  Stark. 


Oval  1:  Ground  Truth 


Oval  2:  Sensor  Coverage 


USS  Stark  is  12  nm  from  Iraqi  exclusion  zone. 

AWACS  is  aloft. 

Other  USN  vessels  in  the  area. 


Oval  3:  Workstation  Display 


USS  Stark  is  12  nm  from  Iraqi  exclusion  zone. 

AWACS  is  aloft. 

Other  USN  vessels  in  the  area. 


Oval  4:  Perception 

Aircraft  on  detected  on  radar. 
Aircraft  tagged  as  friend iy. 


□ 
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Content  of  Lenses: 

US  had  sided  with  Iraqis. 

Brindel  was  on  final  cruise. 

XO  &  TAO  missed  Intel  briefings. 

Iraqis  had  been  flying  farther  south  w/o  incident. 


Oval  5:  Comprehension 


Aircraft  is  no  threat  to  USS  Stark. 


Oval  6:  Projection 

Aircraft  will  turn  away  from  USS  Stark. 


The  crew  comprehend 
the  ship  has  been  hit  but  they 
do  not  know  the  source  of  the  attack. 
Hence,  their  projections  are  of  little  use  to  them. 


Team  Situation  Awareness 


The  degree  to  which  all  team  members  possess 
the  SA  required  for  their  jobs. 


Team 

Goal 


Team  Situation  Awareness 

The  degree  to  which  all  team  members  possess 
the  SA  required  for  their  jobs. 


Team  Goal 


Team 
Situation 
Awareness 


Team  Situation  Awareness 


The  degree  to  which  all  team  members  possess 
the  SA  required  for  their  jobs. 


Tm 

Member 

Two 

Sub-Goal 


Team  Goal??? 
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Team 
Situation 
Awareness 


Shared  Situation  Awareness 


The  degree  to  which  team  members  possess 

the  same  SA. 


Team 

Goal 


No  Shared  Situation  Awareness 


Shared  Situation  Awareness 


The  degree  to  which  team  members  possess 

the  same  SA. 


Team  Goal 


Shared 

Situation 

Awareness 


1 935(M^JZ  also  detects  track 

-  fimS^aoptifres  GMH7cDn  LASALLE 

-  AWAIfiS  tnatttevaluates  as 


-  STARK  CIC  detects  track 

-  STARK  CIC  confirms  track  with 

AWACS 


-  STWahd^iA/aicke^P^  ^roiectioji  -  STARK  CIC  personnel  determine 


Process  Tracing 


□  Maps  out  how  the  incident  unfolded 

□  Focuses  on  how  a  given  outcome  came  about 

□  Externalizes  internal  processes 

□  Uses  data  from  multiple  sources 

□  Describes  the  sequence  of  information  flow  and 
knowledge  activation 


(After  Woods,  1993) 


Metrics  for  Situated  Cognition 


Technological  side  of  model: 

•Playback  of  ground  truth  using  various  sources  (e.g.,  database 
queries,  screen  captures) 

Human  side  of  model: 

•  individuai  characteristics  of  sensemakers 

Experience  and  training  (inciuding  proficiency  with 
computers  and  technoiogy) 

Personaiity,  inteiiect 
Index  of  Learning  Styies 

•Behaviorai  Anaiysis  (direct  observation,  voice  and  AN  Recordings) 
Noidus  System  for  Behaviorai  Capture  and  Anaiysis 

•Physioiogicai  measures  of  participants  (Head  and  Eye  movements, 
heart  rate  variabiiity,  EEGs) 

•Geographicai  Recaii  and  Anaiysis  of  Data  in  the  Environment  (GRADE) 
•Cued  Retrospective  interviews 


Evaluating  Data  Sources 


Subjective 


4^ 


jr  ^ 


>sO 


Objective 


Obtrusive 


(y 


Unobtrusive 


Data 

Dbase  Queries 
Audio/Video 
GRADE 
Retro  Interviews 


Oval 

1,2 

2,  3,  4,  5,  6 
4,  5,6 
4,  5,6 


Data  Oval 
SWAG-C  4, 5 

HR  Monitors  4,  5 

SART  4,5,6 

Wrist  Monitors  4 


GRADE 

(Geographical  Recall  and  Analysis  of  Data  in  the 

Environment) 

•  Not  a  memory  test.  A  way  to  gauge  where  the  officer  is  focused 
at  that  moment. 

•  Facilitates  comparison  between  Oval  3  (what  is  displayed  on  local 
workstation)  and  Oval  4  (what  is  perceived  by  the  decision  maker). 

•  On  cue  participants  turn  away  from  the  screen. 

•  “As  quickly  as  possible,  sketch  the  portion  of  the  battlefield  on 
which  you  are  currently  focused  in  sufficient  detail  to  communicate  it 
to  a  fellow  staff  officer.” 

•  Flip  an  acetate  overlay 

•  “Tell  me  what  the  battlespace  will  look  like  30  minutes  from  now.” 

•  Number  and  timing  of  GRADE  events  based  on  scenario. 


Sample  GRADE 


- - - - —  INSTRUCTIONS 

1 .  Fill  in  your  SART  ratings  below.  (Refer  to  descriptions  on  the  inside  cover  of  this  binder  if  necessary.) 

2.  On  0df3Gr lu&pt  guickly  as  possible,  sketch  the  portion  of  the  Pattisfielci  ort  which  you  are 

currently  focused  in  sufficient  detail  to  communtcate  it  to  a  fellow  staff  officer. 

3.  Flip  the  acetate  overlay. 

4.  On  acefafe  overtav,  sketch  what  the  battlespace  will  look  like  30  minutes  from  now. _ 


Demand  (1  to  7)  Supply  (1  to  7)  tr  Understanding  (1  to  7) 


TT 


Sample  GRADE 


Seat _  Date; _  Time: _ 

INSTRUCTIONS 

1 .  Fill  in  your  SART  ratings  below.  (Refer  to  descriptions  on  the  inside  cover  of  this  birder  if  necessary.) 

2.  On  the  paper  mao,  as  quickly  as  possible,  sketch  the  portion  of  the  battlefield  on  which  you  are 


currently  focused  in  sufficient  detail  to  communicate  it  to  a  fellow  staff  officer. 

3.  Flip  the  acetate  overlay. 

4.  On  the  acetate  overlay,  sketch  what  the  battlespace  will  look  like  30  minutes  from  now. 


GRADE 

(Adapted  for  Trident  Warrior  ’04) 

Experimental  Design 
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4  Data  Pauses  within  each  Segment  (1/2/3)  of  each  Scenario  (A/B/C) 


Ground  Truth 


GRADE 
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Results 


Chart  I  -  progression  of 
GRADE  scores  from 
Scenarios  A  to  B 


Chart  II  -  variation  in 
GRADE  scores  among 

different  watch  stations 
(TAO/ESG/PHIBRON) 
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Sample  Heart  Rate  Variability 


Running  up  ■  ■  ■  ■  ^ 

^  7  min.  low  workload 

and  down  stairs 
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Subjective  Workload  Assessment  Graph  (Cognitive) 

(SWAG-C) 
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Summary 


□  The  case  of  the  USS  Stark  illustrates 
the  utility  of  the  process  model  of 
situated  cognition  as  a  descriptive  and 
explanatory  tool  for  both  individual  and 
collaborative  activities. 

□  The  model  combines  both  human 
and  machine  system  components. 


□  By  employing  multiple  methods  of  data  collection,  the  evolution  of  an 
event  can  be  traced  as  data  and  information  flow  through  the  machine  and 
human  components  of  the  system. 


□  The  model  facilitates  determining  when  and  how  activities  go  awry. 


□  Knowledge  of  how  and  when  errors  occur  is  critical  to  the  design  of 
new  C2  systems  and  the  re-design  of  existing  systems. 
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